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The paper describes the methodology used in an 
ongoing, independent evaluation of the teaching family model, also 
icnovn as "Achievement Place," a community-based group home treatment 
program for predelinquent and delinquent adolescents aged 12-16. 
youths referred by courts and other agencies live vith surrogate 
parents, in groups of eight, for about nine months. T)ie paper 
outlines evaluation procedures aimed at determining relative 
effectiveness of the teaching family program and comparison 
rehabilitation programs. There are seven graphs and charts 
summarizing the criteria used for the comparison (Autt&or/MPB) 
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This paper describes the current status of an ongoing, independent 
evaluation of the Teaching Family Model. The Teaching Family Model, also 
known as Achievement Place, is a community-based, group home treatment 
approach for predelinquent and delinquent adolescents (Phillips, Phillips, 
Fixsen, S Wolf, 1972). A brief description of the Model and its imple- 
mentation throughout the country will provide the necessary background for 
discussion of the independent evaluation approach, some problems in eval- 
uating the Teaching Family Model nationwide, and some preliminary descrip- 
tive data from the first year of field research. 

Description of the Teaching Family Model 

The focus of the evaluation project is the set of Teaching Family 
Model homes which, are similar to the early Achievement Place homes in 
Kansas. The Teaching Family homes are community -based programs, drawing 
youths from the communities in which the homes operate. The youths reside 
in the homes and, in many instances, are assigned to the homes, either 
directly or indirectly, by the juvenile court system in the community. The 
youths participate in the residential group home program, but may attena 
local schools, go to their own churches, engage in various community acti- 
vities, and visit their natural homes on weekends. The five to eight 
youths in each home are under the supervision of a married coupl^e called 
teaching parents. The operation and management of the homes typically 
rests with the teaching parents, who also apply the treatment components 
of the Model. The homes are usually large, older residences which have 
been renovated to meet local codes for use as group homes . The teaching 
parents live in the residence 24- hours a day, typically with their own 
private quarters separate from the living facilities provided for the 
youths . 

The treatment approach of the Teaching Family Model draws on components 
from various approaches that have been used with delinquent and predelinquent 
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youths. While the early research literature on the Teaching Family Model 
focused heavily on the token economy component (e.g. , Phillips, Phillips, 
Fixsen, 6 Wolf, 1971),. this motivation system is only one of a number of 
treatment components. Frequent family conferences are group discussions 
focusing on behavior problems, manageK^ent of the home, and other matters 
which concern the group. Individiial coxanseling between teaching parents 
and youths occurs, and tutorial sessions are used to help youths in their 
scholastic activities. There are prescribed procedures for teaching 
youths new behaviors to replace inappropriate behaviors. 

Teaching parents are trained in the Teaching Family Model via a set of 
workshops plus follow-up mon-ltoring by trainers , resulting eventually in 
certification as teaching p^fXents. The systematic training, monitoring, 
and evaluation procedures tend to produce a relatively homogeneous treat- 
ment approach in all px»ograins staffed by trained teaching parents. 

The youths remain with the program as long as necessary to complete 
the various stages of the treatment model and, barring premature departxires, 
average around nine months' treatment time. Following treatment, youths 
are returned to their nat\iral homes or some surrogate living arrangement. 
While teaching parents are responsible for implementing the youths^ treat- 
ment programs , they work with local agency personnel to facilitate treat- 
ment activities. Each home applies a set of selection criteria for ad- 
mitting youths , and while there is some variation from one coimnxinity to 
another, these selection criteria are sufficiently similar so that the 
youths in different Teaching Family homes represent a fairly homogeneous 
sample of troubled youngsters. 

While some homes have ejcpanded the model to deal with populations such 
as retarded adults, emotionally disturbed younger children, etc., these 
kinds of clients usually are excluded by the homes' selection criteria. 
Generally, the youths in Teaching Family homes are delinquent or predelin- 
quent youths, some are classified dependent/neglected by local community 
agencies, but a.ll have run astray of local community norms and their be- 
havior has brought them inxo contact with agencies ranging from social wel- 
fare departments to the police, probation, and juvenile court systems. 



The Evaluation Project 



The Evaluation Research Group (ERG) is funded for five years; the 
first was a planning year, the second, third, and fotorth are data collec- 
tion years, and the fifth year is for dissemination. At present, the third 
of the five years has just begun. In what follows, the evaluation approach 
for studying the Teaching Family Model and some preliminary descriptive 
data will be presented. 

First, however, consider some rationales for conducting an indepen- 
dent evaluation of the Teaching Family Model. Few would deny the value 
of empirically demonstrating the effectiveness of any innovative delin- 
quency treatment approach. But evaluations by program sponsors, no matter 
how impartially conducted, might be suspected by potential consumers since 
program people may find it difficult to maintain the unbiased perspective 
required for maximally objective research. Outside or independent evalua- 
tors should be better able to maintain a disinterested, impartial attitude 



toward programs which cire not of their own design. Thus, independent 
evaluations can lend credibility to program effectiveness claimed by pro- 
gram sponsors who conduct their own internal evaluation studies. 

Regardless of how this independent evaluation of the Teaching Family 
Model turns out, the very fact that an outside appraisal was conducted will 
enhance the Model ^s stature as a viable treatment alternative to other 
community -based approaches which may be less adequately evaluated. In 
choosing among alternatives, community officials can be expected to select 
treatment models which -have been evaluated over those that have not, even 
if unevaluated models si*em more effective than thoroughly evaltiated ones. 
Limited community resources will dictate placing confidence and money in a 
product about which something is known. The independent evaluation should 
insure that as much will be known about the effectiveness of the Teaching 
Family Model as will be knovm about its competitors. 

Evaluation paradigm . A three-dimensional evaluation paradigm is used 
to organize data collection activities and provide a general framework for 
data analysis. The three dimensions are programs, data collection proce- 
dures, and time. Figure 1 shows the evaltiation model schematically: 
(a) programs are divided into two samples of Teaching Family programs and 
comparison programs (to be described below); (b) the 17 data collection 
procedures are classified into descriptive, process, and outcome kinds of 
measures; and (c) time is partitioned into the three data collection years. 
The paradigm ^s dimensions are fully crossed, meaning that each of the pro- 
grams is assessed with each of the data collection procedxires during each 
of the data collection years. 

While the time dimension shows three data collection years, the acttial 
administration schedules vary among the 17 data collection procedures. 
Some instruments are obtained annually, others more frequently, and others 
are tied to youths' progress through the program. Three years of data 
collection should be stiff icient for adequate follow-up of youths , since 
most youngsters are in treatment for less than one year. Thus, youths 
who enter a treatment program during Year 1 of the evaluation project will 
have at least two years available for follow-up assessment. Similarly, 
youths who enter treatment in Year 2 of our evaluation project would be 
followed for at least one year. Working backwards in time for data collec- 
tion procedures based on existing and accessible records (e.g., court and 
school data), the pre-program time for data collection goes as far back as 
the available records. 

Programs are shown as the xjnits of analysis in Figure 1. An alterna- 
tive paradigm could show the units as either individual youths > program 
staff, or other respondents to questionnaires, interviews, tests, etc. 
But for most planned analyses, .programs will comprise the units of analysis, 
and data obtained from individtial youths, staff, or other persons in the 
field sites will be summarized to obtain measures which will apply to each 
individual program. 

The following sections describe the programs, data collection proce- 
dures, and time dimensions in more detail. Also, discussion will focus on 
specific problems or issues that have occurred or may occur in performing 
the evaluation of the Teaching Family Model. 



Figure 1. 
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Prograins > The cooperation of 25 Teaching Family programs and 31 com- 
parison programs operating in 10 states has been obtained. Teaching Family 
programs were selected to be close replications of the original Achievement 
Place home. Thus, the Teaching Family sample is comprised of homes which 

(a) serve delinquent or pre-delinquent youths, age approximately 12-16, 

(b) are community-based and operated by teaching parents trained, evaluated, 
and/or ceritifed by Teaching Family trainers, and (c) were willing to par- 
ticipate in our evaluation. Teaching Family homes were excluded if they 
served other client populations (e.g*, retarded persons or younger emotion- 
ally disturbed youths) or were heavily committed to other research or 
evaluation activities. All of the Teaching Family homes which met the 
selection criteria for the evaluation agreed to participate when approached 
by the evaluation project staff. This 100% cooperctio:i is unusual in evaliia- 
tion work, and indicates the strong emphasis and interest by the Teaching 
Family Model on evaluation research. Teaching Family programs that have 
started up subsequent to the formation of the evaluation sample wiJJ. not 

be included since sufficient follow-up time would not be available before 
the evaluation project ends. 

The sample of comparison programs was recruited as follows. For each 
community or region from which a Teaching Family program was obtained, a 
list of possible comparison programs was solicited from community agencies. 
These lists comprised the available alternative programs that Teaching 
Family youths might have been placed in had the Teaching Family program 
not been available, excluding some kinds of non-comparable programs, e.g., 
probation or state institutions. In effect, we sought otljer community-based 
programs , typically group homes or some other kind of residential setting 
which served youths similar to those served by Teaching Family homes. 

Evaluation project staff contacted the program staff or directors of 
the possible comparison programs, described the evaluation project, and 
after determining that a program was a suitable comparison, solicited their 
cooperation. In return for their participation, an evaluation report was 
offered which would be submitted to them, for their use only. Altogether, 
approximately 60 comparison programs were asked to participate and coopera- 
tion from 31 (50% acceptance) was obtained. Apparently, the offer of free 
evaluation in this age of accoixnt ability was sufficient inducement to enlist 
the cooperation from programs that otherwise had no reason to participate. 

The 31 comparison programs are more heterogeneous on virtually all 
descriptive dimensions than the 26 Teaching Family programs. Some compari- 
son programs, on the surface at least, appear similar to the Teaching Family 
programs, in that they use a behavioral treatment approach. In contrast, 
other comparison programs use non-behavioral treatment approaches, are 
slightly larger than the typical Teaching Family home, and use different 
staffing patterns. Several potential comparisons fit most of the evalua- 
tion project's criteria, but served youths from many different communities 
or states. Such programs were excluded from the comparison sample simply 
to minimize the cost of collecting court data, from many counties rather 
than only one or two. Since contacting juvenile coixrt judges and obtaining 
their cooperation is often time-consxaming, the fewer different courts re- 
quired in the evaluation, the more efficiently the work can proceed. 
Despite these practical considerations in selecting comparison programs, 
descriptive data reported below suggest that the sample is comparable to 
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the Tea'-.hing Family homes and will allow a satisfactory quasi -experimental 
study of the Teaching Family Model. 

Data collection procedures . The second dimension of the evaluation 
paradigm involves the set of data collection procedures developed to measure 
a variety of descriptive, process, and outcome variables. Table 1 lists 
the 17 data collection procedures and a set of characteristics for each. 
This battery of evaluation instruments is rather comprehensive, yet seems 
suitable for efficient, large-scale field evaluation. If anything, it is 
too extensive in terms of the workload required of field staff. A priority 
order of these data collection procedures could be developed so that if the 
workload becomes a threat to continued cooperation, certain of the lower 
priority measures could be abandoned, either early or late in the evalua- 
tion project. 

The 17 data collection procedures are classified into descriptive, 
process, or outcome kinds of assessments. Descriptive data are necessary 
in evaluation work to understand the salient characteristics of the programs 
being studied. This information is typically demographic, financial, admin- 
istrative, or historical in nature, and provides the background against 
which other kinds of measures and findings can be interpreted. Process 
measures are concerned with implementation and execution of the treatment 
program. Often program evaluation focuses mainly, if not exclusively, on 
outcomes. Without adequate process and/or descriptive study, outcomes 
which may be interesting and important cannot be attributed to specific 
components or features of the treatment program (e.g., Levine, 1973). 
For instance, knowing simply that recidivism (an outcome measure) was re- 
duced by a program is one thing, but knowing the program components that 
contributed most to that reduction is quite another. Process study is 
useful not only for interpreting outcome findings, but also for feedback 
to participating programs after the evaluation is completed, to help pro- 
grams improve their treatment approaches (e.g., Stallings, 1975). In this 
way, program evaluation projects can contribute to the programs in exchange 
for the cooperation obtained during the evaluation. 

Some issues in developing and implementing an extensive data collec- 
tion battery are worth noting. Our assessment approach is m\ilti-dimensional , 
i.e., different variables are measured using as many different instriiments 
and respondents as feasible. A multi-dimensional approach is usefiil for 
obtaining convergence of findings across a variety of measures rather than 
relying on only one or a few measures of effectiveness. Of course, some 
evaluation measures are more important than others. For example, recidivism 
measures are a must in delinquency treatment evaluation studies, but other 
measures of program success should be included, as well, to either lend 
support to or be critical of primary outcome findings such as differences 
in recidivism. But with an elaborate data collection system come problems 
in maintaining the data collection system over several years. Since our 
target programs are scattered throughout 10 states and numerous communities 
within each of those states, project staff have to work long-distance. 
While staff travel extensively, they cannot be on-site at all times and 
must rely on data collectors hired by the evaluation project in the com- 
munities or on program staff for assistance with certain of the measurement 
activities. The use of program staff in data collection raises a potential, 
bias problem, in that the data they collect could be influenced by their 
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statTis as program people. On the other hand, data often are more easily 
collected by program staff with less strain on the program than if special 
arx'angements are necessary to allow evaluation staff to enter the sites for 
data collection. 

The issue of bias, depending on who collects the data, was addressed 
in an early study this past year. We administered some youth instruments 
in two conditions — program staff vs. evaluation staff • A comparison of the 
average scores for youths who were administered the instruments by program 
vs. evaluation staff revealed no differences beyond those expected by 
chance. Hence, for youth self -report instruments like the Tennessee Self- 
Concept Scale, it apparently makes no difference, on the average, whether 
program or evaluation staff collect the data. 

A major problem in extensive evaluation data collection involves 
gaining access to individuals who must provide the data directly or allow 
access to data sources. The most time-consuming and often difficult data 
access problem involves juvenile courts. To cover the 57 programs in this 
study, access to U2 juvenile court systems in the 10 states ^was necessary. 
We have developed a general procedure for approaching juvenile courts to 
gain access to their records, but in the end it is the professional skills 
of evaluation staff that are most important in obtaining approval and con- 
sent from *juvenile court judges to allow access to their records. We have 
approached all 42 juvenile courts, have been turned down by only one, and 
we are still trying to access that one. 

Schools require a different approach and there are different problems 
in accessing their records. But school record data, like court record data, 
are critical outcome measures and must, be accessed. Program staff, who 
often work closely with school personnel, will be relied on to assist in 
accessing school records. Accessing school data will begin in earnest this 
fall, so currently not much inside information on problems with accessing 
school data can be discussed. Preliminarily, however, it has been estab- 
lished that access to school records will be possible in most sites, and 
a data collection system to retrieve information on youths' attendance, 
grades, and disciplinary records has been developed. 

Obtaining all kinds of data, p.articularly court and school records, 
requires strict adherence to procedures which respect individuals' rights 
to privacy Informed consent is obtained from youths, parents, legal 
guardians, and all others who supply data for the study. Courts and schools 
often — and rightfiilly — request copies of signed consent forms before allowing 
access to records. Experience to date suggests that while obtaining parti- 
cipants' consent and gaining access to court or school records may be cum- 
bersome and time consuming, these are not insurmomtable obstacles to con- 
ducting this kind of field evaluation research. 

The time dimension . The third dimension of the evaluation paradigm 
represents the longitudinal component of the project, involving extensive 
repeated measurement of programs and their participants over the three-year 
period. All of the 17 data collection procedures are used more than once 
during the three years and some at relatively frequent intervals, depending 
on youths' progress through the programs. A major issue involves maintaining 
cooperation over the three years, so methods of reinforcing program staff 
for their cooperation are important. Evaluation project staff are as open 
as possible about research activities, and a Bi-Monthly Synopsis of our 
activities is regularly distributed to all persons in the field, even those 
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remotely connected with our project. 

Another problem in this longitudinal study has to do with the effects 
of staff turnover on the ongoing data collection activities • Group home 
staff turnover or "bum-out" is frequent, requiring training new staff in 
the evaluation approaches • New staff cannot be enlisted immediately in 
the evaluation activities because they have enough to do getting into their 
new jobs without being encumbered by an outside evaluation project. To 
facilitate the changeover in our data collection activities, departing 
staff often assist in the training of new staff. Group home programs, of 
course , are not static — they change in a variety of ways , often more fre- 
quently than either staff or evaluators would like, and program staff txim- 
over is one of the most disruptive changes, not only programmatically , but 
from the evaluation standpoint as well. A major project policy is to 
minimize intrusiveness , since program matters must take precedence over 
independent research activities. Disruptions due to staff turnover will 
occasionally result in loss of data due to missed testing schedules • But 
these are the exigencies one encoixaters in field research and evaluators 
must develop a high tolerance, since there is often litte that can be done 
when programmatic disruptions interfere with evaluation activities. 

Description of Teaching Family and Comparison Programs and the Issue of 
Equivalence in Pre-existing Groups 

Evaluation requires comparison, which can come in variolas forms. A 
rudimentary kind of comparison is a simple, pre-post comparison to deter- 
mine if pre-treatment measures change following treatment. This is a weak 
design unless another kind of comparison is included, i.e,, a control or 
contrast treatment (Campbell & Stanley, 1966). True experimental design 
would require random assignment of treatment approaches to two or more 
samples of program units and random assignment of youths to two or more 
kinds .of programs. In the present context, true experimental design would 
require ran(JoTn assignment of the Teaching Family Model and alternative 
treatment approaches to group home programs and random assignment of youths 
to programs , followed by evaluation to determine if the Teaching Family 
Model produced different outcomes from those produced by the alternative 
treatments. Such random assignment of treatment approaches and youths may 
guarantee equal pre-'conditions in the two samples; that is, characteristics 
of the programs, other than treatment approach, would be randomly distri- 
buted in the two sets of programs, p^articularly if the samples are large. 
However, this true experimental design cannot be easily implemented in 
field evaluation studies. Hence, less ideal but more practical quasi- 
experimental designs are tjrpical of natxxral field experiments such as the 
present independent evaluation of the Teaching Family Model. 

The independent evaluation of the Teaching Family Model uses a design 
in which pre-existing programs with ongoing treatment approaches are com- 
pared. The opportunity to randomly assign treatment approaches and youths 
to programs simply did not exist. Since the Teaching Family and comparison 
programs existed intact before the evaluation began, and no attempt was made 
to manipulate their programs experimentally, other pre-existing differences 
between the two groups of programs could occur. And such pre-existing 
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differences could represent threats to the validity of outcome findings. 
For example, if Teaching Family programs served yoxiths which were more 
severely disturbed than the comparison program youths, comparisons between 
youth outcome measures for the two samples of programs could be jeopardized. 
Thus, when field evaluation projects are unable to manipulate the assign- 
ment of treatment approaches to programs, or youths to programs, arid must 
study pre-existing programs with whatever differences already exist between 
samples, it is vital to determine the extent to which the programs differ 
on variables relevant to outcome measures. If the selection of comparison 
programs in this eval'uation project has generated a sample which is similar 
to Teaching Family programs in most or all respects except fox^ expected 
treatment approach differences , then an ideal experimental design has been 
approximated and threats to the validity of outcome comparisons between 
Teaching Family and comparison programs should be minimized. 

While selective sampling of comparison programs may be problemmatic 
for the validity and generalization of results, another matter in sampling 
comparison groups must be considered. Evaluation m'ast be concerned with 
the fairness of comparisons between target programs which have prescribed 
characteristics, and comparison programs, however sampled. A sample of 
comparison programs without the same characteristics as the Teaching Family 
programs , excluding treatment approach , may provide an tmf air comparison 
since different outcomes might be due to differences in the two samples of 
programs other than the different treatment approaches. For example, 
suppose Teaching Family homes were compared with institutional programs. 
Many characteristics relevant to outcomes, in addition to treatment approach, 
could exist, such as size, administrative structure, severity of youths' 
previous history, and so forth. One would expect institutions to be larger, 
with different administrative structures , and to work with more troubled 
youths than community-based group homes. Hence, it would be difficiilt to 
unequivocally interpret differential outcomes for yoxiths served by Teaching 
Family and institutions if such differences in programs existed. Thus, 
unless comparison programs share all or many relevant characteristics with 
the target programs, except treatment approach, comparison of otitcome 
findings could be equivocal. 

These issues guided bur choice of comparison programs , such that the 
Teaching Family homes and the comparison programs would be as similar as 
possible on most characteristics relevant to treatment outcomes, except for 
the treatment approaches used. In the following sections, the samples of 
Teaching Family and comparison programs are described on variables which, 
if the two samples differ, could interfere with comparisons on outcome 
variables. The data are taken from the Program Description Questionnaire, 
which contains information in four general categories: (a) Home and Com- 
munity Characteristics; (b) Program Management and Administration; 
(c) Youth Information; and (d) Treatment Approaches. 

The objective of these preliminary and descriptive data analyses is to 
establish the comparability of the Teaching Family and comparison programs. 
The question to be answered empirically is whether or not the two samples 
of programs differ greatly enough to invalidate outcome comparisons between 
the two samples. The data presented below are based on responses to the 
Program Description Questionnaire provided by staff in the complete sample 
of the 26 Teaching Family programs, but only 13 of the 31 comparison pro- 
grams. To prepare this presentation, it was not possible to include as 
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many comparison programs in the analyses as will be possible later. How- 
ever, preliminary as these dat^ are, thei'e seem to be enough programs in 
both samples to address the is^Ue of pre-existing differences between the 
two samples of programs. 

Home and community charac^ rigt ics . Both Teaching Family and compari- 
son programs are located in coti'^nunities ranging in population from 5,000 
or less to well over 100,000, The median population for both Teaching 
Family and comparisons is between 15,000 and 25,000 people. On the average, 
then, the programs in the two samples are operating in comparably-sized 
communities. 

The local settings of the Programs are described as virbanji suburban, 
or rural neighborhoods. Six of the Teaching Family homes, or 23%, are in 
rural neighborhoods, while six of the comparisons, or 4-6%, are in rural 
neighborhoods, indicating a slight (t>ut statistically non-significant) 
tendency for the comparison programs to be located in rural areas rather 
than urban or siobxirban neighborhoods. 

The size of the programs is described by the number of youths that the 
program is licensed to accommodate. Here, the range is from four to 10-plus 
in both samples , and the median number of youths is eight for both Teaching 
Family and comparison programs* On "the average, then, the two samples of 
programs serve the same numbers of youths, in tersns of licensing. 

Other descriptive characte^^istics of the programs and communities 
showed a similar absence of differences in the two samples. The physical 
facilities were comparable on items such as n^imbers of bedrooms , recrea- 
tional facilities, staff office space, separate staff living quarters, 
access to recreational facilities in the community, and socio-economic 
status of the neigfiborhood. ThUs, at this early stage in the evaluation, 
it is fair to conclude that the Teaching Family and comparison programs do 
not differ in any important respects with i'»egard to their home or community 
characteristics. 

Organizational and adminis_trative strtgcture . Table 2 shows the per- 
centage of programs in the two samples which reported how financial matters 
and administrative policies ar^ operationalised and by whom. Because the 
Teaching Family approach tends to place much of the program *s financial 
and administrative responsibility on the teaching parents, the significant 
differences obtained for these two questions are expected. There was a 
clear trend toward greater involvement of treatment staff in financial and 
administrative matters in the Teaching Family programs. For example, 
teaching parents in 42% of the Teaching Family homes managed the financial 
matters, while none of the compaa^ison. program staff managed financial 
matters ♦ In contrast, the board of directors has more financial involvement 
in the comparison nrograms (^6%) than in the Teaching Family programs (12%). 
Similarly, the boards in the comparison pJ^grams are much more influential 
in determining administrative policies (85%) than in the Teaching Family 
programs (19%). 

Another area of concern in operating community-based group homes in- 
volves the amount of outside influence on various aspects of the programs. 
Table 3 reports staff responses in the two samples regarding outside .in- 
fluences .on .treatment approach* administrative policies, personnel policies, 
- and-f inancial-"mattersT-'"--The -frequenoies-and -percentag 

"Yes," "No," and "No response/' in "the two samples are shown in Table 3, 
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None of the four items produced statistically significant chi squares, so 
there is no evidence favoring one sample of programs over the other. Well 
over half of the programs in both samples indicated no outside influence on 
treatment approaches, administrative or personnel policies, and financial 
matters . 



Frequency and Percen"^ of Teaching Family (TF) and 
Comparison (Comp.) Programs* Responses 
for Organizational Questions 









Who manages your 


Who determines yoiir 






home's financial 


home's administrative 






matters? 






policies? . 










T- 


•F 


Comp. 


T- 


•F 


Comp. 






f 


% 


f 


% 


f 


% 


f 


% 


1. 


Treatment staff 


11 


42 


0 


0 


6 


24 


0 


0 


2. 


Board of directors 


3 


12 


6 


46 


5 


19 


11 


85 


3. 


Admiuistrative staff 


3 


12 




31 


7 


26 


0 


0 


4. 


Local agency- 


2 


8 


0 


0 


0 


0 


0 


0 


5. 


CD + (2) + (3) 


7 


26 • 


3 


23 


8 


31 


2 


15 


Total 


26 


100 


13 


100 


26 


100 


13 


100 



Note: Chi-squares for both questions were significant (£ < .01) when 
eight response categories were used. These eight were reduced 
to the five shown to facilitate discussion. 



Responsibility for selection of youths into programs could vary among 
programs. Table ^ shows that in both samples, about half report that a 
selection committee has the ultimate responsibility for youth selection, 
and better than 60% involve staff or program directors in youth selection. 
Boards of directors and chiefs of local social services have considerably 
less responsibility for youth selection than do selection committees and/or 
program staff. The absence of significant differences suggests compara- 
bility between the two samples of programs regarding responsibility for 
youth selection. 

The next two items in Table 4, however, do show significant ^differences 
between the Teaching Family and comparison programs. First, we inquired 
about the extent to which the programs used volunteers. Thirty-eight per- 
cent of the Teaching Family programs reported, "Yes," while 77% of the 
comparisons did, a statistically significant finding (x = 6.3; £^ < .05). 

. Tiiis,..difference^is .expected>.„giy„en-i:he^^ 

Family programs, where sxibstantial responsibility for all aspects of the 
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Table 3 

Frequency and Percent of Program Responses for the Question, 
"Are there any persons or organizations outside of 
your home's staff or board of directors that 
can directly influence any of the following?" 





T-F 
f % 


Comp. 

f % 


a) Treatment approach 

Yes 
No 

No response 


8 31 
18 69 
0 0 


5 3'8 
7 54 
1 8 


Total 


26 100 


13 100 


b) Administrative policies 

Yes 
No 

No response 


6 23 
19 73 
1 4 


1 8 
11 84 
1 8 


Total 


26 100 


13 100 


c) Personnel policies 

Yes 
No 

No response 


5 19 
20 77 
1 4 


1 8 
11 84 
1 8 


Total 


26 100 


13 100 


d) Financicil matters 

Yes 
No 

No response 


9 35 
17 65 
0 0 


3 23 
9 69 
1 8 


Total 


26 100 


13 100 
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Table 

Responses for Teaching Family and Comparison Programs to 
Questions about Youth Selection, Use of Volunteers, 

and Evaluation 

a) Which of the following has the ultimate responsibility for selection 
of youths into your program? (percentages, where more than one 
answer could app'' 

T-F Comp. 

1) Selection c... .^e 58% • **6% 

2) Board of directors 19 8 

3) Staff or program director 65 62 

4) Chief of social services 15 23 

b) Do you use volunteers in your program? (x^ = 6,3; £ < •05) 

T-F Comp. 



f 


% 


f 


% 


10 


38 


10 


77 


15 


58 


2 


15 


1 


«♦ 


1 


8 


26 


100 


13 


100 



Yes 
No 

No response 

Total 

c) Is program evaluation included as part of your progratn? (x^ - 6.5; 
£ < .05) 

T-F Comp . 



Yes 
No 

No response 

Total 



f 


% 


f 


% 


26 


100 


10 


77 


0 


0 


2 


15 


0 


0 


1 


8 


26" 


100 


13 


100 
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program is given to the teaching parents. This is less true of the com- 
parison programs and marks one important difference in the treatment 
approaches of the programs in the two samples. The third item in Table 
also shows significant differences between the two samples . In answer to 
the question about program evaluation being included in their program3 , 
100% of the Teaching Family programs reported "Yes while 77% of the com- 
parison programs responded "Yes" (x^ = 6.5; £_ < .05). VOiile the samples 
differed significantly, programs in both samples place heavy emphasis on 
some form of internal program evalioation. Even in this age of accoxxnta- 
bility, this strong emphasis on program evaluation in these programs could 
account for the cooperation that this evaluation project has enjoyed in 
working wt-^^ jjrograms. 

"on. The two samples ip ilude programs with co-ed living 
arrange, vu^, as v^ell as boys only or girls only. Sixty-one percent (16 
of 26) of the Teaching Family programs and 54-% (seven of 13) of the com- 
parison programs are boys only. Teaching Family programs are more likely 
to serve girls only than comparisons (27% Teaching Family and 8% compari- 
sons). However, comparison programs are more often co-ed (38%) than 
Teaching Family (12%). These differences are not statistically significant, 
indicating comparability in ntmibers of programs dealing with boys only, 
girls only, or having co-ed arrangements. Also, the number of youths served 
by the programs is the same for both Teaching Family and comparison samples. 
At the time our Program Description Questionnaires were collected, the 
median number of youths in the two samples of programs was five. 

The median age range for youths acceptab^r into the Teaching Faaiily 
and comparison programs are also quite compar: 3. For the Teaching 
Family programs , the median minimum age is i: ad the onedian maximim age 
is 16. Hbr the compari a ansy the median miniE age is 11, and the median 
maximisn:^ 17. Henc*:^^ H heTO ; is only one year .fference im the median age 
range :fcrr youths betisreen The two san^sles — U : 16 for Teaching Family and 
11 to r~ for comparisons.. 

-djiquired abciut tim referral so\irces £imv miich the programs had 
receivaa youths. Table 5 lists the referral saarces and the results for 
Teaching Family and comparison programs. Sincsr more than one referral 
source could be checked, the percentages do norz: add to 100%. The diffe- 
rences in percentages between the two samples for each referral source are 
non-significant, although there are some trends worth noting. Somewhat 
higher percentages were reported for welfare referrals in the Teaching 
Family programs than the comparisons, 85% versus 5H%, respectively. For 
an evaluation of delinquency treatment programs, it is important that the 
juvenile court referral soxirce be comparable between the two samples. Very 
close percentages were obtained — 85% Teaching Family and 92% comparison 
for juvenile court referrals. The other results in Table 5 do not show 
important differences , so it can be concluded that the referral soxirces 
for the two samples of programs are quite comparable. 

In addition to referral sources , acceptance criteria for youths is an 
important potential difference between Teaching Family and comparison pro- 
grams. We inquired whether programs would accept youths described in 
certain general ways. Table 6 lists the descriptors and the frequencies 
and percentages for the Teaching Family and comparison programs. None of 
~the"nil:fferences""in^percentages~were^statisticaily-signi-fica^^ — 
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showed interesting trends. Teaching Family homes, for e3<ample, are more 
likely to accept adjudicated youths (92%) than are comparisons (77%). On 
the other hand, comparisons are more likely to accept violent offenders 
(62%) than are Teaching Family homes (H2%). Despite these statistically 
non-significant differences in acceptance criteria, these trends suggest 
a tendency for the comparison programs to serve slightly more troubled 
youths than the Teaching Family programs. This possible difference between 
the two samples will be more thoroughly studied when court record data on 
youths become available. 



Table 5 

Percent of Prog.dms for the Question, "Please check which of 
the following have ever referred youths to your home?" 







T-F 


Comp. 


a) 


Welfare 


85% 


5»f% 


b) 


Juvenile Court 


85 


92 


c) 


Ministers 


0 


3 


d) 


Maturi-l ~BT^n~Tr 


9 


7 


e) 


Mental —-i-^. 


12 


8 


f) 


Other 


7 


^ 



Table 6 

Percent rr^^Errr^ams that Accept Youths Described as Follows 







T-F 


Comp. 


a) 


Adjudic-HSE. 


92% 


77% 


b) 


Violent : , hhs'-hvs 


1*2 


62 


c) 


Status () "r^n'ft^-s 


100 


85 


d) 


Dependeii .iHPiyl^icted 


85 


69 


e) 


Non-ad j ir: • : .vr^-r ed 


51+ 


77 


f) 


Emotionally disturbed 


85 


69 



Treatment approaches . The first section of data from the Program 
Description Questionnaire involves questions about (a) the theoretical 
orientations underlying treatment approaches, (b) the community -based 
character of the programs, and (c) the length of stay in program for 
youths . 

Table " sem^ ^percentages of programs which assigned first ranks to 
the various- -rsaffiHEHsat approaidies. The behavioral or token economy treatment 
approach was^ 'Mtiy gi first by 80% of the Teaching Family homes. In contrast, 
53% of the ( iLidL^rf i son pro grans ranked a behavioral or token economy approach 
first. SinT?e Teaching Family Model per se was not listed as a treatment 
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approach, it seems fair to assume that the Teaching Family programs ranK^d 
either the behavioral or token economy approaches to indicate the Teaching 
Family Model, The 53% first rankings for comparison programs is interesting 
and perhaps suggests an increasing prevalence of behavioral orientations in 
group homes generally. An eclectic approach was ranked first for 8% of the 
Teaching Family programs and 31% for the comparison programs. The eclectic 
label used in Table 7 identifies the "other" option provided for the ques- 
tion, where most respondents who ranked the "other" category first vnr^te 
in phrases such as "eclectic" or "combination of the above," Finally, 
reality therapy, humanistic, psychoanalytic, and guided group interaction 
were not prevalent first choices of treatment approach in either sample* 
No first rankings were obtained in either sample of programs for app3?oaches 
identified in the questionnaire such as Gestalt, traditional casework, tranS' 
actional analysis, Dreikxarsian, or Summerhill approach. 



Table 7 



Most Descriptive Treatment Approaches (Ranked First) 
for Teaching Family and Comparison Programs 



Treatment approach 


T-F 


Comp. 


f 


% 


f 


% 


Behavioral or token economy 


21 


80 


7 


53 


Behavioral 


11 


42 


2 


15 


Token economy- 


10 


38 


5 


38 


Eclectic 


2 


8 


^■ 


31 


Reality therapy 


0 


0 


1 


8 


Humanistic 


2 


8 


0 


0 


Psychoanaly-tic 


0 


0 


1 


8 


Guided group interaction 


1 


^■ 


0 


0 


Totals 


26 


100 


13 


100 



Note: No first rankings were obtained from either Teaching Family or 
comparison programs for Gestalt, traditional casework, transac- 
tional analysis, Dreikiirsian, or Summerhill approaches. 



The findings in Table 7 suggest that the theoretical orientations^ 
underlying treatment approaches do not differ between the Teaching ^Family 
and comparison programs as much as one might have expected. Behavioral- 
treatment orientations are clearly prevalent in these group homes, both 
for Teaching Family and for comparison programs as well. From the stand- 
point of evaluation research design, one might have hoped that fewer com- 
parisons would rank behavioral approaches first,. so that treatment approach 
differences between Teaching Family and comparison programs would be greater 
than they appear from the data in Table 7, However, wher using pre^ejcisting 
groups in field res earch, one mu st take things as they come, ^^t ^^e^Toajn 
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focus of the evaluation project is on the Teaching Family Model versus com- 
parison programs, not on behavioral versus non-behavioral treatment 
approaches. And, of course, there are no Teaching Family Model programs 
in the comparison sample , even though some comparison programs may espo\ise 
a behavioral treatment orientation. In sum, we are confident that the 
comparison programs are sufficiently unlike the Teaching Family programs 
in treatment approach so that outcome differences, if any, between the two 
samples can be attributed to differences in the* treatment approaches , i.e.. 
Teaching Family programs versus non -Teaching Family programs. 

The community-based characteristics of the programs also are relevant 
aspects of treatment approaches, particularly since the Teaching Family 
Model emphasizes community involvement, while other treatment ^grsr 
not to the same extent Table 8 shows the results for three iteto concerned 
with youths* acceptability to the program, i.e. , being able to attend pijblic 
schools, to retxam home on weekends, and to have a natrural home available 
in the coirmunity. The results show significant differences for each of 
the three items, st^gesting that Teaching Family homes more often require 
that acceptcd)le youths be able to attend public schools, go home on week- 
ends, and have natural homes in the community than is required for the 
comparison programs. For example, 85% of the Teaching Family homes indi- 
cated tiiat attendance at public schools was necessary for youths to be 
accepted into the program, while only 31% of the comparisons did so. 
Similarly, 69% of the Teaching Family homes considered going home on week- 
ends a requirement for acceptable youths, while only 8% of the comparison 
programs did. Finally, Teaching Family programs more often required that 
the youth's natxiral hcame be in the same community, although only 31% of 
the Teaching Family programs indicated "Yes," compared to none of the 
coiiiparison programs. The overall picture from these three irems is that 
the Teaching Family Model., more than the comparisons, focuses on community 
access for youths with regard to schooling and their natural family situa- 
tion- These findings fit with the Teaching Family treatment approach, 
which emphasizes these very points in its approach to helping youths from 
the commxmity return to the community following treatment. 

Finally, we inquired about youths' length of stay in the programs. 
The average length of stay for Teaching Family youths was about nine months 
(N = 21), while for comparisons it was eight months (N = 9). The average 
minimum stay in pjpogram was three months for Teaching Family programs and 
two months for comparison programs. And, the average maximimi length of 
stay in program was 17 months for Teaching Family and 18 months for com- 
parisons. None of these differences in average, minimimj, or maximum length 
of stay was statistically significant between the two samples, indicating 
comparability between Teaching Family and comparison programs in youths'" 
length of stay. 



Concliision 



In sum, these preliminary Program Description Questionnaire results 
siuggest satisfactory comparability between the Teaching Family and com- 
-parisonrTjrograms— '-This--comparabii^^^ 
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Table 8 

Responses to Questions about Cotnraunity-based Selection Criteria 
for Acceptable Youths 

a) Acceptable youths must be able to: 

T-F Comp . 

f % f % 

1) ''ttend public schools 



Yes 


22 


85 


4 


31 


No 




15 


8 


61 


No response 


_0 


0 


J. 


8 


Total 


26 


100 


13 


100 



(X^ = 11.8; £_ < .01) 



2) Go home on vjeekends 



Yes 


18 


69 


1 


8 


No 


8 


31 


11 


84 


No response 




_0 


_1 


8 


Total 


26 


100 


13 


100 



(X^ = 13.9; 2. < -01) 



b) Must youth's natural home be in the same coTnmumity as your program? 



Yes 


8 


31 


0 


0 


No 


18 


69 


12 


92 


No response 


_a 


0 


'J: 


8 


Total 


26 


100 


13 


100 



(X^ = 6.6; 2^ < .05) 
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design requirement that the two samples of programs do not differ impor-. 
.tantly on features of their programs which coxild influence outcome findings. 
The item differences Which do exist, e.g. , staff responsibility for finan- 
cial and administrative matters, are expected sir: hese qu#*«stif)ns t-- 
information which characterizes the Teaching Fam:L,^^ Model an nriy not ^^c 
terize comparison progr>ams. Fun ler study of this ■ ax^al ..xo. ^ ^vH^§lion 
will be pursued as more complete data are available, but from these prelimi- 
nary results, it can be tentatively concluded that the two samples of pro- 
grams shoxild provide a fair comparison for evaluating the effectiveness of 
the Teaching Family Model. 
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